Control scheme for a scanning display

ABSTRACT

Described is a display system comprising a light source, a scanning assembly, and a controller. The light source includes a first plurality of emitters forming a column that emits a first color. The light source also includes a second plurality of emitters forming a column that emits a different color. The scanning assembly includes a reflective surface rotated by the controller using a control signal having a nonlinear, pulsed waveform. The controller drives the first plurality of emitters and the second plurality of emitters in sequence, during respective portions of the nonlinear waveform, and in synchronization with rotation of the reflective surface such that light emitted by the first plurality of emitters and the second plurality of emitters is reflected onto a same pixel of an output image. The controller also adjusts emission durations of individual emitters to compensate for changes in the rotational speed of the reflective surface.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Non-Provisional applicationSer. No. 17/204,807, filed Mar. 17, 2021, titled “Control Scheme For AScanning Display,” which is a continuation of U.S. Non-Provisionalapplication Ser. No. 16/274,143, filed Feb. 12, 2019, titled “ControlScheme For A Scanning Display,” which claims the benefit and priorityunder 35 U.S.C. 119(a)-(d) of Greece Patent Application No. 20190100026,entitled “Control Scheme For A Scanning Display,” filed Jan. 11, 2019,the contents of each of which are incorporated by reference herein intheir entirety for all purposes.

BACKGROUND

Most displays, such as computer monitors, smartphone screens, andtelevision screens, are viewed directly. Displays are sometimesincorporated into a head mounted device that includes optical elementswhich operate on image light produced by the display to form an outputimage for viewing by a user. There is usually a one-to-onecorrespondence between the pixels of a display image and the pixels ofthe output image seen by the user. In contrast, a scanning display formsan output image by combining multiple display images using a scanningmirror, e.g., to form a larger composite image using multiple, smallerdisplay images. Scanning displays are useful in applications where, dueto space constraints, a full sized display would be impractical. Becausethe scanning mirror needs to be synchronized with the outputting of thedisplay images, the timing of the scanning should be taken intoconsideration as well as the timing with which image data is supplied tothe display. The timing requirements for operating the scanning mirrorare not always consistent with the timing requirements for operating thedisplay.

BRIEF DESCRIPTION OF THE DRAWINGS

Illustrative embodiments are described with reference to the followingfigures.

FIG. 1 shows a waveguide assembly for implementing one or moreembodiments.

FIG. 2 is a simplified block diagram of a display system forimplementing one or more embodiments.

FIG. 3 shows the operation of the scanning display of FIG. 2 .

FIGS. 4A to 4E show shifting of image data from a scanning display ontoa user's eye, for controlling a scanning display in at least someembodiments.

FIGS. 5 to 8 show example waveforms usable for controlling a scanningassembly, in accordance with some embodiments.

FIG. 9 shows a rising portion of a sinusoidal waveform for controlling ascanning assembly, in accordance with an embodiment.

FIG. 10 shows a division of emission periods among different portions ofa waveform, in accordance with an embodiment.

FIG. 11 is a flowchart of a method for operating a display system, inaccordance with an embodiment.

The figures depict embodiments of the present disclosure for purposes ofillustration only. One skilled in the art will readily recognize fromthe following description that alternative embodiments of the structuresand methods illustrated may be employed without departing from theprinciples, or benefits touted, of this disclosure.

In the appended figures, similar components and/or features may have thesame reference label. Further, various components of the same type maybe distinguished by following the reference label by a dash and a secondlabel that distinguishes among the similar components. If only the firstreference label is used in the specification, the description isapplicable to any one of the similar components having the same firstreference label irrespective of the second reference label.

The terms “row” and “column” are used herein to refer to a physicalarrangement of emitters and/or emitter related circuitry into groups,and are sometimes used together to differentiate between two spatialdimensions that are orthogonal to each other. Rows and columns aregenerally interchangeable and should not be taken to refer to anyparticular dimension. For instance, a row can refer to either thehorizontal or the vertical dimension of a display device.

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, specificdetails are set forth in order to provide a thorough understanding ofcertain inventive embodiments. However, it will be apparent that variousembodiments may be practiced without these specific details. The figuresand description are not intended to be restrictive.

Example embodiments relate to techniques for controlling a displaysystem that includes a scanning display. In particular, exampleembodiments are directed to controlling a scanning assembly insynchronization with output of images on a display. Embodiments aredescribed in connection with a scanning assembly in which the scanningelement is a scanning mirror driven using one or moremicroelectromechanical systems (MEMS) components. For example, thescanning mirror can be rotated in one or more dimensions using one ormore MEMS actuators. However, the techniques described herein can beapplied for controlling other types of scanning elements. In general,any reflective surface capable of scanning across a display can becontrolled in accordance with the techniques herein.

Embodiments of the invention may include or be implemented inconjunction with an artificial reality system. Artificial reality is aform of reality that has been adjusted in some manner beforepresentation to a user, which may include, e.g., a virtual reality (VR),an augmented reality (AR), a mixed reality (MR), a hybrid reality, orsome combination and/or derivatives thereof. Artificial reality contentmay include completely generated content or generated content combinedwith captured (e.g., real-world) content. The artificial reality contentmay include video, audio, haptic feedback, or some combination thereof,and any of which may be presented in a single channel or in multiplechannels (such as stereo video that produces a three-dimensional effectto the viewer). Additionally, in some embodiments, artificial realitymay also be associated with applications, products, accessories,services, or some combination thereof, that are used to, e.g., createcontent in an artificial reality and/or are otherwise used in (e.g.,perform activities in) an artificial reality. The artificial realitysystem that provides the artificial reality content may be implementedon various platforms, including a head-mounted display (HMD) connectedto a host computer system, a standalone HMD, a mobile device orcomputing system, or any other hardware platform capable of providingartificial reality content to one or more viewers.

FIG. 1 shows a waveguide assembly 100 for implementing one or moreembodiments. In some embodiments, the waveguide assembly 100 is acomponent of a near-eye display (NED), for example, an HMD. Thewaveguide assembly 100 includes a scanning display 110, an outputwaveguide 120, and a controller 130. For purposes of illustration, FIG.1 shows the waveguide assembly 100 associated with a single eye 190, butin some embodiments, another waveguide assembly separate (or partiallyseparate) from the waveguide assembly 100, provides image light toanother eye of the user. In a partially separate system, one or morecomponents may be shared between waveguide assemblies for each eye.

The scanning display 110 generates image light 155. The scanning display110 includes a light source 140 and an optics system 145. The lightsource 140 is an optical component that generates light using aplurality of emitters placed in an array.

The optics system 145 performs a set of optical processes, including,but not restricted to, focusing, combining, collimating, transforming,conditioning, and scanning processes on the image light generated by thelight source 140. The optics system 145 may include a conditioningassembly and a scanning mirror assembly, which are shown in FIGS. 2 and3 . The scanning display 110 generates and outputs image light155—influenced by at least one of the light source 140, the conditioningassembly, and the scanning mirror assembly—to one or more couplingelements 150 of the output waveguide 120.

The output waveguide 120 is an optical waveguide that outputs images tothe eye 190 of the user. The output waveguide 120 receives the imagelight 155 at one or more coupling elements 150, and guides the receivedinput image light 155 to one or more decoupling elements 160. In someembodiments, the one or more coupling elements 150 couple the imagelight 155 from the scanning display 110 into the output waveguide 120.The one or more coupling elements 150 may include, e.g., a diffractiongrating, a holographic grating, some other element that couples theimage light 155 into the output waveguide 120, or some combinationthereof. For example, in embodiments where the coupling elements 150include a diffraction grating, the pitch of the diffraction grating ischosen such that total internal reflection occurs, and the image light155 propagates internally toward the one or more decoupling elements160.

The one or more decoupling elements 160 decouple the total internallyreflected image light from the output waveguide 120. The one or moredecoupling elements 160 may include, e.g., a diffraction grating, aholographic grating, some other element that decouples image light outof the output waveguide 120, or some combination thereof. For example,in embodiments where the one or more decoupling elements 160 include adiffraction grating, the pitch of the diffraction grating is chosen tocause incident image light to exit the output waveguide 120. Anorientation and position of the light exiting from the output waveguide120 is controlled by changing an orientation and position of the imagelight 155 entering the one or more coupling elements 150.

The output waveguide 120 may be composed of one or more materials thatfacilitate total internal reflection of the image light 155. The outputwaveguide 120 may be composed of e.g., silicon, plastic, glass, orpolymers, or some combination thereof. The output waveguide 120 has arelatively small form factor. For example, the output waveguide 120 maybe approximately 50 mm wide along an x-dimension, 30 mm long along ay-dimension and 0.5 to 1 mm thick along a z-dimension.

The controller 130 controls the scanning operations of the scanningdisplay 110. The controller 130 determines scanning instructions for thescanning display 110 based at least on the one or more displayinstructions. Display instructions are instructions to render one ormore images. In some embodiments, display instructions may simply be animage file (e.g., bitmap). The display instructions may be receivedfrom, e.g., a console of a NED system (not shown). Scanning instructionsare instructions used by the scanning display 110 to generate imagelight 155. The scanning instructions may include, e.g., a type of asource of image light (e.g., monochromatic or polychromatic), a scanningrate, an orientation of a scanning apparatus, one or more illuminationparameters (described below with reference to FIG. 2 ), or somecombination thereof. The controller 130 includes a combination ofhardware, software, and/or firmware not shown here so as not to obscureother aspects of the disclosure.

FIG. 2 is a simplified block diagram of a display system 200 forimplementing one or more embodiments. The display system 200 includes ascanning display 210, which is an embodiment of the scanning display 110of FIG. 1 , and further includes a controller 230, a light source 240,and an optics system 250. The light source 240 is an embodiment of thelight source 140; the optics system 250 is an embodiment of the opticssystem 145; and the controller 230 is an embodiment of the controller130.

The scanning display 210 generates image light 245 in accordance withscanning instructions from the controller 230. The scanning display 210includes a light source 240 and an optics system 250. The light source240 is a source of light that generates a spatially coherent or apartially spatially coherent source light 215. The source light 215 maycomprise a display image. The optics system 250 comprises at least aconditioning assembly 270 and a scanning assembly 280. The conditioningassembly 270 conditions the source light 215 into conditioned light 235,and the scanning assembly 280 scans the conditioned light 235. The imagelight 245 may be coupled to an entrance of an output waveguide (e.g.,one or more coupling elements 150 of the output waveguide 120 of FIG. 1).

The light source 240 emits light in accordance with image data in theform of one or more illumination parameters received from the controller230. An illumination parameter is used by the light source 240 togenerate light. An illumination parameter may include, e.g., sourcewavelength, pulse rate, pulse amplitude, beam type (continuous orpulsed), other parameter(s) that affect the emitted light, or somecombination thereof. The illumination parameter can be applied to anemitter of the light source 240 using analog and/or digital signals. Theillumination parameter and/or other image data can be supplied from thecontroller 230 to circuitry that generates, based on the image data, thesignals which drive the light source. This driving circuitry can beincluded in the light source 240 (e.g., co-located with emitters of thelight source) or located external to the light source 240.

The light source 240 comprises a plurality of emitters, wherein eachemitter may be, e.g., a light-emitting diode (LED), a laser diode, avertical cavity surface emitting laser (VCSEL), an organic LED (OLED), amicro-LED (uLED), a tunable laser, or some other light source that emitscoherent or partially coherent light. The emitters of the light source240 emit light in a visible band (e.g., from about 390 nm to 700 nm),and they may emit light in accordance with one or more illuminationparameters. In some embodiments, the scanning display 210 comprisesmultiple light sources each with its own array of emitters emittinglight in a distinct wavelength such that when scanned, light emittedfrom each of the light sources are overlapped to produce variouswavelengths in a spectrum. Each emitter of the light source 240comprises an emission surface from which a portion of source light isemitted. The emission surface may be identical for all emitters or mayvary between emitters. An emitter width is a width of an area of theemission surface. The emission surface may have different shapes (e.g.,circular, hexagonal, etc.). For example, an emitter which is a uLED witha circular emission surface may have an emitter width of 25 micrometerscharacterized as a diameter of the circular emission surface.

The plurality of emitters of the light source 240 is arranged as anarray of emitters. The emitters can be organized in a one-dimensional(1D) or two-dimensional (2D) array. In a 2D array, the emitters areformed along a first dimension and a second dimension orthogonal to thefirst dimension (e.g., along rows and columns). Each column of emitterscorresponds to a respective column in an image ultimately displayed tothe user. The emitters may be of various colors. For example, the lightsource 240 may include a set of red emitters, a set of green emitters,and a set of blue emitters, where emitters of different color togetherform an individual pixel. An individual pixel may include at least onered emitter, at least one green emitter, and at least one blue emitter.Rows of emitters of the same color may be arranged in a single group.For example, the array may comprise N rows of red emitters followed by Nrows of green emitters and then N rows of blue emitters.

The light source 240 may include additional components (e.g., drivingcircuits, memory elements, heat sinks, etc.). In one or moreembodiments, the light source 240 comprises a plurality of data shiftingcircuits and a plurality of driving circuits, which are electricallycoupled to the array of emitters. The data shifting circuits may supplyimage data from the controller 230 to the driving circuits, which thengenerate signals that activate the emitters. In particular, as explainedin connection with FIGS. 4A to 4E, image data can be sequentiallyshifted through a row or column of emitters to form a display image,with the resulting emitted light being scanned to form an output image.The driving circuits include circuitry for controlling the array ofemitters based on the image data. For example, the driving circuits mayapply illumination parameters received from the controller 230 (e.g.,brightness values received from a display driver of the controller) tocontrol each emitter in the array of emitters using analog and/ordigital control signals. The emitters can be controlled using currents(i.e., the display can be a current mode display) or voltages. In someembodiments, the emitters are controlled using pulse-width modulation(PWM), amplitude adjustments, or a combination of both.

The conditioning assembly 270 conditions source light 215 from the lightsource 240. Conditioning the source light 215 may include, e.g.,expanding, collimating, focusing, distorting emitter spacing, adjustingorientation an apparent location of an emitter, correcting for one ormore optical errors (e.g., field curvature, chromatic aberration), someother adjustment of the light, or some combination thereof. Theconditioning assembly 270 comprises one or more optical elements (e.g.,lenses, mirrors, apertures, gratings, or any other suitable opticalelement that affects image light).

The scanning assembly 280 includes one or more optical elements thatredirect light via one or more reflective portions of the scanningassembly 280. The reflective portions may comprise a scanning mirror orother reflective surface. The direction where the light is redirectedtoward depends on specific orientations of the one or more reflectiveportions. The one or more reflective portions of the scanning assemblymay form a planar or curved surface (e.g., spherical, parabolic,concave, convex, cylindrical, etc.) that operates as a mirror. Thescanning assembly 280 scans along at least one dimension of a 2D emitterarray. In some embodiments, the scanning assembly 280 is configured toscan in at least the smaller of the two dimensions. For example, if theemitters are arranged in a 2D array where the rows are substantiallylonger (i.e., contain more emitters) than the columns, then the scanningassembly 280 may scan down the columns (e.g., row by row or multiplerows at a time). In other embodiments, the scanning assembly 280 mayperform a raster scan (horizontally or vertically depending on scanningdirection). The scanning assembly 280 can include multiple scanningmirrors, each of which is configured to scan in 0, 1, or 2 dimensions.The scanning can be controlled using one or more MEMS devices, e.g.,electrostatic or electromagnetic actuators, included in the opticssystem 250.

The controller 230 controls the light source 240 and the optics system250. The controller 230 takes content for display and divides thecontent into discrete sections. The controller 230 instructs the lightsource 240 to sequentially present the discrete sections usingindividual emitters corresponding to a respective row or column in animage ultimately displayed to the user. The controller 230 instructs oneor both of the conditioning assembly 270 and the scanning assembly 280to condition and/or scan the presented discrete sections. The controller230 controls the optics system 250 to direct the discrete sections ofthe image light 245 to different areas, e.g., different portions of oneor more coupling elements 150 of the output waveguide 120. Accordingly,at the eye box of the output waveguide, each discrete portion ispresented in a different location. While each discrete section ispresented at different times, the presentation and scanning of thediscrete sections occurs fast enough such that a user's eye integratesthe different sections into a single image or series of images. Thecontroller 230 also provides illumination parameters (e.g., intensity orbrightness values) for the light source 240. The controller 230 maycontrol each individual emitter of the light source 240.

The controller 230 may include a combination of software and/or hardwarecomponents that control the scanning assembly 280 in synchronizationwith controlling the light source 240. For example, the controller 230may include one or more computer processors, a dedicated graphicsprocessor, application-specific integrated circuits, software programscontaining instructions for execution by the one or more computerprocessors, etc. In some embodiments, the controller 230 includes adisplay driver 232 and a separate MEMS controller 234. The displaydriver 232 can be implemented as an integrated circuit that generatesthe image data for the light source 240 based on instructions from acomputer processor executing a software application that determines thedisplay images. For example, the software application can be anapplication that generates an AR or VR presentation for viewing on anHMD. The MEMS controller 234 may include circuitry that generatescontrol signals for one or more MEMS devices that drive the scanningassembly 280. The control signals can include periodic waveforms withlinear or sinusoidal pulses. The display driver 232 and the MEMScontroller 234 may be communicatively coupled to one another tofacilitate the synchronization of output from the display driver 232with output from the MEMS controller 234. In some embodiments, thecontroller 230 includes timing circuitry such as clock generator thatproduces one or more clock signals which determine the timing of theoutputs of the display driver 232 and the MEMS controller 234. The clocksignals may, for example, determine various operational phases for theoutput of instructions to the light source 240 and/or the output ofinstructions to the MEMS devices.

FIG. 3 shows the operation of the scanning display 210 of FIG. 2 . Thescanning display 210 generates light in accordance with scanninginstructions from the controller 230. The light source 240 of thescanning display 210 generates the spatially coherent or the partiallyspatially coherent source light 215. The optics system 250 receives thesource light 215 and with the conditioning assembly 270 converts thesource light 215 into conditioned light 235. The conditioned light 235is then scanned by the scanning mirror assembly 280. The scanningassembly 280 may perform the scanning by rotating about one or more axes(e.g., an axis 310), thereby emitting the image light 245 in one or moredimensions.

FIGS. 4A to 4E illustrate shifting of image data 400 from a scanningdisplay onto a user's eye. The process shown in FIGS. 4A to 4E can beused to control a scanning display in accordance with the techniquesdescribed herein. The image data 400 is depicted using alphabeticallabels “A” to “M”, with each letter representing a row of image data.Each piece of image data 400 is transmitted over time to N number ofemitters 412. The emitters 412 may belong to different rows of the samecolumn. Alternatively, the emitters 412 may belong to different columnsof the same row. The shifting process depicted in FIGS. 4A to 4E can beused to simultaneously drive multiple rows/columns at a time. Forexample, in one embodiment the scanning display has 2,560×1536 emitterswith 3 colors (e.g., red, green, and blue) and N=8 rows per color. Oneway to operate the emitters is to send image data for all N emittersevery time the row/column is activated. For example, the image data A,C, and E could be sent to emitters 412-C, 412-B, and 412-A,respectively, followed by image data B, D, and F during the next rowtime. The process of FIGS. 4A to 4E shows an alternative method thatreduces the amount of data that needs to be sent.

FIG. 4A shows image data G being loaded into the emitter 412-A andscanned by an optics system 410 to emit light onto an output image 420.The output image 420 corresponds to an image projected onto an eye of auser. As shown, the value G is represented in the output image 420 at aspatial location corresponding to the location of the value Gin theimage data 400. For convenience, these locations will be referred to aspixels even though, as explained earlier, a pixel may include multipleemitters of different colors rather than a single emitter. Each pixel inthe output image 420 may be illuminated N times for each color over ascan cycle. For example, the image data G may be output N times, eachtime using a different one of the emitters 412. To avoid retransmittingthe image data each time, the image data can be stored using storageelements 414 and shifted into the next emitter that is to receive thesame image data.

FIG. 4B shows the image data G transferred to storage element 414-A andloading of new image data H into the emitter 412-A. The image data H isprojected onto a corresponding pixel in the output image 420.

FIG. 4C shows the image data G loaded into emitter 412-B from thestorage element 414-A. The image data H is loaded into the storageelement 414-A in preparation for loading into the emitter 412-B duringthe next row time. Additionally, new image data I is loaded into emitter412-A, with the image data G and I being projected onto the output image420.

FIG. 4D shows the image data H loaded into emitter 412-B from thestorage element 414-A. The image data I is loaded into the storageelement 414-A in preparation for loading into the emitter 412-B duringthe next row time. Similarly, the image data G is loaded into thestorage element 414-B in preparation for loading into the emitter 412-Cduring the next row time. Additionally, new image data J is loaded intoemitter 412-A, with the image data H and J being projected onto theoutput image 420.

FIG. 4E shows the image data G loaded into emitter 412-C from thestorage element 414-B and the image data I loaded into emitter 412-Bfrom the storage element 414-A. The image data H and the image data Jare loaded into the storage elements 414-B and 412-A, respectively.Additionally, new image data K is loaded into emitter 412-A, with theimage data G, I, and K being projected onto the output image 420. Thus,FIGS. 4A to 4E illustrate the sequential loading of new image data intoa first emitter (i.e., emitter 412-A), with the image data being shiftedinto other emitters (i.e., emitters 412-B and 412-C) using the storageelements 414, until the end of the row or column is reached. It can beseen that if the process were to continue, each item of image data wouldbe projected onto the output image 420 a total of N times, once for eachemitter 412, each time corresponding to a different rotational positionof a scanning assembly. Each pixel of the output image 420 would then beperceived as an aggregate of N number of brightness units.

Example waveforms for controlling a scanning assembly through a MEMSdevice will now be described. The waveforms can be applied to activate aMEMS device to trigger a rotational movement of a scanning assembly insynchronization with control of a display. In particular, the displaycan be controlled so that the emitters emit light during specificperiods of time relative to the rotational movement of the scanningassembly. The example waveforms are described as having portions (e.g.,rising or falling portions of individual pulses) or segments (e.g., asegment within a particular rising or falling portion) that correspondto emission times or emission periods. For discussion purposes, it isassumed that the waveforms can be applied to effect instantaneousmovement of the scanning assembly, such that the timing of therotational movements exactly matches that of the waveforms. Thus, thewaveforms are used to describe the rotational movements. However, it isunderstood that in practice, there may be a slight delay from when awaveform is applied to when the scanning assembly begins to rotate.There may also be delays when reversing a direction of rotation. Suchdelays can be caused, for example, by inertia of the MEMS device and/orinertia of the reflective surface.

FIG. 5 is a graph of a waveform 500 that can be used to drive a MEMSdevice for controlling a scanning assembly, in accordance with anembodiment. The waveform 500 is a linear waveform comprising a set ofpulses that repeat periodically (at a particular frequency). Thewaveform 500 is shown with a frequency of 120 Hertz (Hz). The waveform500 represents a clock signal that can be output from the controller 230to control a MEMS device of the optics system 250. Each pulse includes arising portion 510 that ramps up linearly as a function of time,followed by a corresponding falling portion 520 that is also linear.When emitting on a rising portion, the rising portion is generallylarger in duration than the falling portion. The optics system 250 mayinclude circuitry configured to drive the MEMS device so that thescanning assembly rotates across a range of scan angles. In the exampleof FIG. 5 , the scan angles range from −20 to +20 degrees. The zerodegree position may correspond to a position at which the optical axisof the reflective surface of the scanning assembly is orthogonal to thecenter of the display. For example, if the display and the scanningassembly both include flat, 2D surfaces, then the zero position can be aposition at which both surfaces are parallel to each other. As shown,each pulse includes a zero crossing on the rising portion as a well as azero crossing on the falling portion. Because the rising portions 510are linear, the speed at which the scanning assembly rotates across therange of scan angles is constant. Similarly, the speed at which thescanning assembly rotates during the falling portions 520 is alsoconstant.

As mentioned earlier, the scanning assembly is driven in synchronizationwith the display. For example, the display can be controlled such thatlight is emitted on only the rising portions 510, with the fallingportions corresponding to periods of non-emission. In that case, therate at which image data is supplied to the display (i.e., the framerate) would be 120 Hz (corresponding to a frame period of 8.3milliseconds) with a duty cycle of 80%. The amount of time spentsupplying data for a particular row of emitters is referred to herein asthe row time. In general, the row time is equal to the emission timedivided by the number of rows.

FIG. 6 is a graph of a waveform 600 that can be used to drive a MEMSdevice for controlling a scanning assembly, in accordance with anembodiment. Unlike the waveform 500, the waveform 600 is non-linear. Inparticular, the waveform 600 is a sinusoidal waveform that can be usedto drive a MEMS device according to a resonant mode of operation (e.g.,causing an electrostatic actuator to oscillate at a particularfrequency). Resonant MEMS devices provide certain benefits over linearMEMS devices, including lower power consumption, reduced size, andpotentially larger scan angles. However, as will be explained,controlling a resonant MEMS device in synchronization with a display canbe challenging.

The waveform 600 is shown with a frequency of 240 Hz, with the range ofscan angles being approximately the same as in FIG. 5 . If the displayis controlled to emit on every rising portion 610 of the waveform 600(i.e., so that emission times correspond to rising portions andnon-emission times correspond to falling portions), then the frame ratewould be 240 Hz. Because the rising portions 610 are non-linear, therotational speed of the scanning assembly is not constant. At the zerocrossing of the rising portions 610, the speed is higher compared to thebeginning or end of the rising portion. The speed difference depends onthe starting point of the rising portion and the end point of the risingportion. For example, if the top and bottom 10% of each pulse is clipped(as discussed below), then the speed at the zero crossing of the risingportions 610 could be 2.3 times higher compared to the speed at thestarting point (i.e., the beginning of the non-clipped part of therising portions 610) or the speed at the end of the non-clipped part ofthe rising portions 610. Therefore, the rate at which image data issupplied for each row of the display should also be non-constant. Forexample, each row of the display could be driven for a particular amountof time within a range between a minimum row time and a maximum rowtime, where the average row time is equal to the emission time dividedby the number of rows. Emission durations can also be adjusted for eachrow in correspondence with changes in row time. In particular, theemission duration of a row can be decreased for shorter row times andincreased for longer row times. If the row times and emission durationswere constant, this could lead to incorrect mapping of display imagesonto the output image. For example, since faster speeds result in moredisplay area being covered in any given amount of time, failure toadjust the timing of the display images could lead to non-uniform pixelsizing of the output image, i.e., non-uniform resolution. Therefore, thetiming of the image data should be dynamically adjusted so that lesstime is spent supplying image data at faster movement speeds and moretime is spent at slower speeds. The frame rate and the MEMS frequencyare therefore interrelated.

One option for reducing the complexity of the circuitry for driving thedisplay when a resonant MEMS based scanning assembly is used would be toconfine the emission times to the most linear segment of the risingportion 610. The most linear segment is at the center of the risingportion 610. In comparison with the center segment, the beginning andend segments of the rising portion 610 are much more non-linear.Accordingly, part of the beginning and end of each rising portion 610could be ignored for emission purposes. For example, 10% of the top andbottom portions of every pulse can be clipped off and ignored when usingthe waveform 600 for determining the timing of emission. However, theexact amount of clipping can vary and the amount by which the top isclipped can be different than the amount by which the bottom is clipped.

FIG. 7 is a graph of a waveform 700 that can be used to drive a MEMSdevice for controlling a scanning assembly, in accordance with anembodiment. The waveform 700 has a frequency of 480 Hz. However, unlikewith the waveform 600, the emission periods correspond to every otherrising portion 710, i.e., a frame rate of 240 Hz. The decision not toemit during every pulse can be based on timing constraints on theoperation of the scanning assembly and/or timing constraints on theoperation of the display. For example, the scanning assembly may berestricted to operation above a certain resonant frequency (e.g., 400 Hzor more) for mechanical stability reasons (hence the choice of 480 Hzfor the resonant frequency). Thus, the design of the scanning assemblymay dictate the range of frequencies with which the MEMS devices can bedriven.

Additionally, the display system is bandwidth limited in that the amountof data that can be supplied in a given amount of time from thecontroller to the circuitry that drives the emitters is finite (hencethe choice of a 240 Hz frame rate). A higher frame rate would require acorrespondingly higher data bandwidth. Further, in this example,although the frame rate is the same as in FIG. 6 (240 Hz), there is lesstime to drive the emitters in any given frame because the durations ofeach pulse of the waveform 700 are shorter than those of the waveform600. For example, when driving the display in conjunction with waveform700, the average row time could be 0.48 microseconds and the emitteron-time per frame could be 0.74 milliseconds (in contrast to therespective times mentioned above for the waveform 600: 0.97 microsecondsand 1.49 milliseconds). In practice, there may need to be a compromisebetween stable resonant operation and efficient data transfer. Forexample, after selecting a frame rate based on the design of the displaysystem, a resonant frequency at which the scanning assembly can operatestably can be selected.

As with the waveform 500, because the waveform 600 is non-linear, therow times and emission durations for the display should be dynamicallyadjusted to account for variation in the speed of the scanning assembly.In addition or as an alternative to the clipping technique describedearlier, other techniques for reducing the impact of speed variationscan be applied. These additional techniques are described below.

FIG. 8 shows a waveform 800 that can be used to drive a MEMS device forcontrolling a scanning assembly, in accordance with an embodiment. Thewaveform 800 has a frequency of 480 Hz. Similar to the example of FIG. 7, the emission times of the display are limited to every other risingportion. Thus, an emission period 810 corresponds to a first risingportion, followed by a non-emission period 820 including a sub-period825 corresponding to the next rising portion. The emission period 810can be allocated for driving different color emitters. For example, in adisplay comprising N rows of red emitters followed by N rows of greenemitters and then N rows of blue emitters, the emission period 810 canbe divided into a first period 830 in which the red emitters areactivated, a second period 840 in which the green emitters areactivated, and a third period 850 in which the blue emitters areactivated. The periods 830, 840, and 850 may be equal in duration sothat each color is driven for one-third of a full row time (e.g., anaverage per color row time of 160 nanoseconds, for an average row timeof 480 nanoseconds).

FIG. 9 shows a rising portion 900 of a sinusoidal waveform. The risingportion 900 corresponds to an emission period that has been divided intothree sub-periods 910, 920, and 930. As mentioned earlier, the centersegment of the rising portion is the most linear segment and thus hasthe least amount of speed variation. This corresponds to the sub-period930. The beginning and end segments are more non-linear and thereforehave more speed variation. While the earlier described clippingtechnique can be applied to constrain the emission time as much aspossible to the center segment, restricting emission to only thesub-period 930 would greatly reduce the amount of time that can be spentdriving the display and, consequently, may require a higher databandwidth than is supported by the display system. Therefore, the extentto which clipping can be applied is limited. One way to make use of thesub-periods 910 and 920 while mitigating the effects of speed variationis to move one or more sub-periods so that the sub-periods 910 to 930 donot all correspond to the same rising portion or pulse, as shown in FIG.10 .

FIG. 10 shows a waveform 1000 for which the emission periods correspondto every rising portion. Emission on every other rising portion is alsopossible, as is emission on falling portions instead of rising portions.Emission can even be performed on both rising and falling portions, aslong as the display system can operate fast enough. As shown in FIG. 10, only part of every rising portion is used for emission. In particular,the beginning and end segments are used for emission periods 1010 and1020 on every other rising portion, alternating with the use of thecenter segments for emission periods 1030. The emission periods 1010,1020, and 1030 correspond to the sub-periods 910, 920, and 930 in FIG. 9, respectively. Thus, emission can be performed in two phases: a firstphase in which the beginning and end segments are used, and a secondphase in which the center segment is used. These two phases correspondto different pulses, with the phases being repeated in alternatingfashion.

The division of the emission time as shown in FIG. 10 is advantageousbecause the most uniform portion of the scan range is isolated from theless uniform portions. Further, although the beginning and end segmentsare non-linear, they are symmetric with respect to each other and thusthe speed variations for the emission periods 1010 and 1020 areapproximately the same. The data for emission periods 1010, 1020 canalso be loaded at a different time than the data for emission period1030, thereby reducing bandwidth consumption. In this manner, thecircuitry that drives the emitters can settle at one frequency for thefirst phase and then have time to prepare for operation at a differentfrequency for the second phase. In comparison to dynamically adjustingfor speed variation across the entire scan range, (1) it becomes easierto balance light output (e.g., the control scheme for adjusting emissiondurations of each row can be simplified) and (2) the lower bandwidthconsumption means that more time is available for generating andsupplying image data to the display. The emission periods 1010 and 1020do not have to be equal in duration to the emission period 1030.Instead, the emission time can be divided according to the speed of thescanning assembly, which may depend on the shape of the waveform (e.g.,the slopes of the rising portion). For instance, the emission periods1010 and 1020 could each occupy 30% of the rise time while the emissionperiod 1030 occupies 20% of the rise time.

The emission time can be further divided, for example, by splitting eachof the emission periods 1010 and 1020 into two parts, splitting theemission period 1030 into three parts, and forming additional phases forthe new parts in accordance with the grouping shown in FIG. 10 (i.e.,pairing emission times for less linear segments together, with separateemission phases for more linear segments). Additionally, the emissiontime can be allocated equally among emitters of different colors, asdiscussed earlier in connection with FIG. 8 . For example, the first setof emission periods 1010 to 1030 can be used for driving red emitters,then the next set of emission periods 1010 to 1030 can be used fordriving green emitters, followed by a set of emission periods 1010 to1030 for driving blue emitters.

FIG. 11 is a flowchart of a method 1100 for operating a display systemthat includes a scanning display, in accordance with an embodiment. Themethod 2100 can be performed using the display system 200. At step 1110,a control signal including a periodic, non-linear waveform is applied todrive the rotation of a scanning assembly. The control signal may, forexample, be applied to cause one or more MEMS devices to oscillate afrequency matching that of the control signal, which in turn causes ascanning mirror or other reflective surface coupled to the MEMS devicesto rotate through a range of scan angles at the same frequency.

At step 1120, one or more portions at the beginning or end of each pulsein the waveform is excluded from being used for light emission. Forexample, as described earlier, the top 10% and bottom 10% of each risingportion and/or pulse can be clipped for emission purposes. Thecontroller 230 can output a non-clipped version of the waveform fordriving the scanning assembly 280 while forming a clipped version of thewaveform for use in determining emission times.

At step 1130, the emitters are driven during one or more less linearsegments of a first pulse (e.g., during the emission periods 1010-A and1020-A in FIG. 10 ). The emission durations of each row and the rate atwhich image data is supplied to the circuitry that drives each row canbe adjusted according to the rotation speed during this time. Forexample, the display driver 232 may be configured to perform a firstalgorithm that dynamically adjusts the row time during the emissionperiod 1010-A (e.g., through appropriate timing of row selection signalsand corresponding data signals), then perform a second algorithm thatimplements an inverse row timing during the emission period 1020-B. Thealgorithms may be part of a control scheme designed specifically for usein conjunction with rotational movements produced in response toapplication of the less linear segments. The control scheme can beimplemented in various ways including, for example, through a lookuptable or based on a mathematical function that describes the angularposition of the scanning assembly as a function of time. In someembodiments, the control scheme is based on the shape of the waveform.For example, the emission durations can be varied based on changes inthe slope of the waveform, since slope is indicative of linearity. Theslope can be calculated for a particular segment of a pulse (e.g., asegment of a rising or falling portion), as a slope of a tangent line ofthe control signal at a particular point in time, or based on multipleslopes of the control signal (e.g., an average slope along a particularsegment). In general, emission durations are adjusted in inverserelationship to changes in slope. For example, when the slope decreases,the emission durations can be increased. When emission durations areincreased, the rate at which data is supplied to each row can becorrespondingly decreased since more time is available for sending datato the emitter drive circuitry.

At step 1140, the emitters are driven during a more linear segment of asecond pulse (e.g., during the emission period 1030-A). As with step1130, the rate at which image data is supplied to the circuitry thatdrives each row can be adjusted according to the rotation speed duringthis time. Since the rotation speed is more uniform in comparison thespeed during the less linear segments, fewer adjustments may be needed.In some embodiments, the display driver may apply, for the more linearsegment, a control scheme that assumes that the more linear segment iscompletely linear (e.g., by setting a uniform emission duration forevery row based on the average slope of the more linear segment). Steps1120 to 1140 can be repeated while continuing to apply the controlsignal to drive the scanning assembly, until one or more complete outputimages have been formed. The above described adjustments to emissiondurations and data rates can be performed irrespective of whether theemission times are divided among different phases and irrespective ofwhether the waveform is clipped. In general, whenever a non-linearcontrol signal is applied to effect rotation of a scanning assembly,emission timing can be dynamically adjusted to compensate for changes inrotation speed.

The foregoing description of the embodiments of the disclosure has beenpresented for the purpose of illustration; it is not intended to beexhaustive or to limit the disclosure to the precise forms disclosed.Persons skilled in the relevant art can appreciate that manymodifications and variations are possible in light of the abovedisclosure.

Some portions of this description describe the embodiments of thedisclosure in terms of algorithms and symbolic representations ofoperations on information. These algorithmic descriptions andrepresentations are commonly used by those skilled in the dataprocessing arts to convey the substance of their work effectively toothers skilled in the art. These operations, while describedfunctionally, computationally, or logically, are understood to beimplemented by computer programs or equivalent electrical circuits,microcode, or the like. Furthermore, it has also proven convenient attimes, to refer to these arrangements of operations as modules, withoutloss of generality. The described operations and their associatedmodules may be embodied in software, firmware, and/or hardware.

Steps, operations, or processes described may be performed orimplemented with one or more hardware or software modules, alone or incombination with other devices. Although the steps, operations, orprocesses are described in sequence, it will be understood that in someembodiments the sequence order may differ from that which has beendescribed, for example with certain steps, operations, or processesbeing omitted or performed in parallel or concurrently. In someembodiments, a software module is implemented with a computer programproduct comprising a computer-readable medium containing computerprogram code, which can be executed by one or more computer processorsfor performing any or all of the steps, operations, or processesdescribed.

Embodiments of the disclosure may also relate to an apparatus forperforming the operations described. The apparatus may be speciallyconstructed for the required purposes, and/or it may comprise ageneral-purpose computing device selectively activated or reconfiguredby a computer program stored in the computer. Such a computer programmay be stored in a non-transitory, tangible computer readable storagemedium, or any type of media suitable for storing electronicinstructions, which may be coupled to a computer system bus.Furthermore, any computing systems referred to in the specification mayinclude a single processor or may be architectures employing multipleprocessor designs for increased computing capability.

The language used in the specification has been principally selected forreadability and instructional purposes, and it may not have beenselected to delineate or circumscribe the inventive subject matter. Itis therefore intended that the scope of the disclosure be limited not bythis detailed description, but rather by any claims that issue on anapplication based hereon. Accordingly, the disclosure of the embodimentsis intended to be illustrative, but not limiting, of the scope of thedisclosure, which is set forth in the following claims.

What is claimed is:
 1. A method for forming an output image, comprising:rotating a reflective surface using a control signal, wherein thecontrol signal is a nonlinear waveform comprising pulses that repeat ata particular frequency, and wherein a rotational speed of the reflectivesurface depends on a slope of the pulses in the control signal; drivinga first plurality of emitters and a second plurality of emitters insequence, wherein: the first plurality of emitters forms a column ofemitters configured to emit light of a first color, the second pluralityof emitters forms a column of emitters configured to emit light of asecond color different from the first color, the first plurality ofemitters is driven during a different portion of the nonlinear waveformthan the second plurality of emitters, and the driving is performed insynchronization with the rotating of the reflective surface such thatlight emitted by the first plurality of emitters and the secondplurality of emitters is reflected onto a same pixel of the outputimage; and adjusting emission durations of individual emitters in thefirst plurality of emitters and the second plurality of emitters tocompensate for changes in the rotational speed of the reflectivesurface.
 2. The method of claim 1, wherein the driving of the firstplurality of emitters and the second plurality of emitters comprises:driving the first plurality of emitters during a first part of a firstpulse; and driving the second plurality of emitters during a second partof the first pulse, wherein a duration of the first part is equal to aduration of the second part.
 3. The method of claim 2, wherein the firstpart and the second part correspond to a rising or falling portion ofthe first pulse.
 4. The method of claim 1, wherein the driving of thefirst plurality of emitters and the second plurality of emitterscomprises: driving the first plurality of emitters during a first partof a first pulse and a second part of a second pulse; and driving thesecond plurality of emitters during a third part of a third pulse and afourth part of a fourth pulse, wherein a combined duration of the firstpart and the second part is equal to a combined duration of the thirdpart and the fourth part.
 5. The method of claim 4, wherein the firstpulse and the second pulse precede the third pulse and the fourth pulse.6. The method of claim 4, wherein the first part is less linear than thesecond part, and wherein the third part is less linear than the fourthpart.
 7. The method of claim 4, wherein the first part comprises twoless linear portions of the first pulse, and wherein the second partcomprises a more linear portion of the second pulse.
 8. The method ofclaim 4, wherein the first part and the third part are identical inshape, and wherein the second part and the fourth part are identical inshape.
 9. The method of claim 1, wherein the adjusting of the emissiondurations of individual emitters in the first plurality of emitters andthe second plurality of emitters comprises: increasing an emissionduration when the reflective surface is rotating at a slower rotationalspeed, relative to an emission duration when the reflective surface isrotating at a faster rotational speed.
 10. A display system, comprising:a light source including a first plurality of emitters and a secondplurality of emitters, wherein the first plurality of emitters forms acolumn of emitters configured to emit light of a first color, andwherein the second plurality of emitters forms a column of emittersconfigured to emit light of a second color different from the firstcolor; a scanning assembly including a reflective surface facing thelight source, the reflective surface being rotatable in at least onedimension; and a controller configured to: rotate the reflective surfaceusing a control signal, wherein the control signal is a nonlinearwaveform comprising pulses that repeat at a particular frequency, andwherein a rotational speed of the reflective surface depends on a slopeof the pulses in the control signal; drive the first plurality ofemitters and the second plurality of emitters in sequence, wherein thecontroller drives the first plurality of emitters during a differentportion of the nonlinear waveform than the second plurality of emitters,and wherein the controller performs the driving in synchronization withrotation of the reflective surface such that light emitted by the firstplurality of emitters and the second plurality of emitters is reflectedonto a same pixel of an output image; and adjust emission durations ofindividual emitters in the first plurality of emitters and the secondplurality of emitters to compensate for changes in the rotational speedof the reflective surface.
 11. The display system of claim 10, whereinto drive the first plurality of emitters and the second plurality ofemitters, the controller is configured to: drive the first plurality ofemitters during a first part of a first pulse; and drive the secondplurality of emitters during a second part of the first pulse, wherein aduration of the first part is equal to a duration of the second part.12. The display system of claim 11, wherein the first part and thesecond part correspond to a rising or falling portion of the firstpulse.
 13. The display system of claim 10, wherein to drive the firstplurality of emitters and the second plurality of emitters, thecontroller is configured to: drive the first plurality of emittersduring a first part of a first pulse and a second part of a secondpulse; and drive the second plurality of emitters during a third part ofa third pulse and a fourth part of a fourth pulse, wherein a combinedduration of the first part and the second part is equal to a combinedduration of the third part and the fourth part.
 14. The display systemof claim 13, wherein the first pulse and the second pulse precede thethird pulse and the fourth pulse.
 15. The display system of claim 13,wherein the first part is less linear than the second part, and whereinthe third part is less linear than the fourth part.
 16. The displaysystem of claim 13, wherein the first part comprises two less linearportions of the first pulse, and wherein the second part comprises amore linear portion of the second pulse.
 17. The display system of claim13, wherein the first part and the third part are identical in shape,and wherein the second part and the fourth part are identical in shape.18. The display system of claim 10, wherein to adjust emission durationsof individual emitters in the first plurality of emitters and the secondplurality of emitters, the controller is configured to increase anemission duration when the reflective surface is rotating at a slowerrotational speed, relative to an emission duration when the reflectivesurface is rotating at a faster rotational speed.
 19. A non-transitorycomputer readable storage medium storing instructions that, whenexecuted by one or more processors of a computer system, cause the oneor more processors to: rotate a reflective surface using a controlsignal, wherein the control signal is a nonlinear waveform comprisingpulses that repeat at a particular frequency, and wherein a rotationalspeed of the reflective surface depends on a slope of the pulses in thecontrol signal; drive a first plurality of emitters and a secondplurality of emitters in sequence, wherein: the first plurality ofemitters forms a column of emitters configured to emit light of a firstcolor, the second plurality of emitters forms a column of emittersconfigured to emit light of a second color different from the firstcolor, the first plurality of emitters is driven during a differentportion of the nonlinear waveform than the second plurality of emitters,and the driving is performed in synchronization with rotation of thereflective surface such that light emitted by the first plurality ofemitters and the second plurality of emitters is reflected onto a samepixel of an output image; and adjust emission durations of individualemitters in the first plurality of emitters and the second plurality ofemitters to compensate for changes in the rotational speed of thereflective surface.
 20. The non-transitory computer readable storagemedium of claim 19, wherein to adjust emission durations of individualemitters in the first plurality of emitters and the second plurality ofemitters, the instructions cause the one or more processors to increasean emission duration when the reflective surface is rotating at a slowerrotational speed, relative to an emission duration when the reflectivesurface is rotating at a faster rotational speed.